Multi-modal Interface with Voice and Head Tracking for Multiple Home Appliances

In this paper, we describe a multi-modal interface that allows use of voice and gesture commands for controlling distributed home appliances used by people with disabilities. The main objective of this study is combined with nonverbal and verbal interface for intuitive and efficient control that uses hands-free operation. The pointing gesture by facing as nonverbal interface represents selecting one of the home appliances. The voice commands as verbal interface represent button operation of the remote controller such as the power on/off, the channel select and the volume up/down. The prototype system can provide a hands-free remote controller for people with quadriplegia who do not have to send verbal commands for selecting home appliances.